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Abstract. In this report, we discuss the problem of computing the zeta func- 
tion of an algebraic variety defined over a finite field, with an emphasis on 
computing the reduction modulo p m of the zeta function of a hypersurface, 
where p is the characteristic of the finite field. 
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1. Introduction 

Let p be a prime number. Let ¥ q be a finite field of q elements of characteristic 
p. Let X be an algebraic variety defined over ¥ q , say an affine variety defined by 
the vanishing of r polynomials in n variables: 

/i(xi, • •• ,x n ) = ••• = f r (xi,- ■■ ,x n ) = 0, 

where the polynomials /j have coefficients in F 9 . Let N(X) denote the number of 
F g -rational points on X. 

Problem I. Compute N(X) efficiently. 

In addition to its intrinsic theoretical interest, this fundamental algorithmic 
problem has important applications in diverse areas such as coding theory, cryp- 
tography, primality testing, sphere packing, quasi-random number generator and 
fast multiplication. It is also very useful in bounding the number of torsion points 
of an abelian variety over a number field. 

In principal, one can always check all the q n possibilities for (x%, • ■ • , x n ). This 
is apparently very slow if either q or n is large. One would like to have faster 
non-trivial algorithms. In this paper, we discuss only deterministic algorithms in 
theoretical sense and we shall frequently use less precise but intuitive description. 

For varieties with large automorphism groups (such as diagonal hypersurfaces), 
it is possible to find efficient elementary methods (such as Gauss sums) to compute 
N(X). We shall, however, restrict to those methods which have potential to extend 
to general varieties. 

In the special case that X is an elliptic curve 

E : y 2 = x 3 + ax + b, a,b G ¥ q , 
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a polynomial time algorithm was obtained by Schoof [Scl]. More practical but 
probabilistic versions were obtained later by Atkin, Elkies, Couveignes and a few 
other authors, see [Sc2] for an updated exposition. Schoof 's algorithm was gen- 
eralized to abelian varieties and curves by Pila [Pi] with some improvements by 
Adleman-Huang [AH]. Curves and abelian varieties are the cases which have been 
studied most extensively in the literature. For more general varieties, no non-trivial 
algorithm is known unless the variety is defined over a small subfield of ¥ q . It is 
shown in [GKS] that this problem is #P-hard (at least NP-hard) for sparse plane 
curves if one uses sparse input size. Thus, in order to get efficient algorithms to 
compute N(X), one might need to put some restrictions on some of the parameters 
of X/¥ q , such as the dimension, degree, characteristic p, etc. 

For each positive integer k, let ¥ q k be the finite field of q k elements. This is 
the unique extension field of ¥ q of degree k. Let Nk(X) denote the number of 
¥ q k -rational points on X. We shall consider the following harder problem. 

Problem II. Compute the sequence {Nk(X)}^L 1 efficiently. 

For fixed n, Problem I is easy if q is small. One can simply check all q n 
possibilities for x &¥ q . On the other hand, Problem II can already be hard even 
for small q as q k will be large for large k. Nevertheless, one can still hope to find 
efficient algorithms in important cases that arise from various applications. 

The sequence Nk(X) has a nice structure. In fact, by Dwork's theorem [Dw], 
the generating zeta function 



^N k (X)^ 



Z(A,T) = cxp(^^-^T fc ) 

fc=i K 

is a rational function. Thus, the sequence N k (X) satisfies a linear recurrence rela- 
tion. Write 

for some polynomials g(T),h(T) G Z[T]. Taking logarithmic derivative, one finds 
VvmT*-i a'(T) h'(T) 

Thus, Problem II is equivalent to 

Problem III. Compute the zeta function Z(X,T) efficiently. 

If an efficient algorithm to compute N(X) is known, then one can continue to 
compute Nk(X) for all positive integers k up to something like the "degree" of the 
zeta function Z(X,T). This easily leads to what I called the naive algorithm to 
compute Z(X, T). It works well if the degree of Z(X, T) is small and if one knows 
how to compute N{X) efficiently In particular, one obtains a good algorithm if 
the degree of Z(X,T) is small and if X is defined over a small subfield of ¥ q (q 
can be large). But the degree of the zeta function can be very large. For a smooth 
projective hypersurface of dimension n and large degree d, the degree of the zeta 
function is about the size d n+1 . 

For an elliptic curve E, the zeta function is determined by Ni(E): 

1 + c(E)T + qT 2 



Z(E,T) 



(\-T)(l-qT) ' 
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where c(E) — N\(E) — (q + 1). Thus, the zeta function Z(E,T) can be computed 
efficiently by Schoof 's work. More generally, when X is an abelian variety of a fixed 
dimension embedded in a fixed projective space, the zeta function Z(X, T) can also 
be computed efficiently by the work of Pila and Adleman-Huang. 

For a non-singular plane curve of large degree d over a small finite field ¥ q (q = 2 
or 3 for instance), computing N\(X) is easy since q is small. Even in this special 
case, computing Z(X, T) already seems to be difficult as Katz-Sarnak pointed out 
[Po]. This case was motivated by their investigation of the distribution of the 
zeros and poles of zeta functions. The results of Adleman-Huang give a doubly 
exponential algorithm for a general plane curve, although they have a significantly 
faster algorithm for hyperelliptic curves (still exponential in terms of the degree). 
This is far from being practical if the degree is large. 

Computing the zeta function of a hyperellitptic curve is very useful in the hyper- 
elliptic curve cryptosystem proposed by Koblitz [Kol-2]. Additional applications 
may be found in [AH2] , [AG] and [Ts] . Computing zeta functions of algebraic vari- 
eties should also be the first key step in computing zeta functions of more general 
Hilbert sets and definable sets, see [Wal] and [FH J] . A very special case is already 
considered in [GKS]. 

Our purpose here is to give a brief general discussion of the modular approach 
for computing zeta functions, indicating some theoretical tools that are available 
and some realistic results that one might hope. As supporting evidence, we shall 
also describe some elementary theorems on computing the reduction of Z(X, T) 
modulo p m . 

2. Modular approach for computing zeta functions 

Let X be an algebraic variety defined over a finite field ¥ q . The degree of 
Z(X, T) can be estimated explicitly using p-adic methods as done by Bombieri 
[Bo]. The size of the coefficients in Z(X, T) can also be bounded easily using trivial 
estimate. In nicer cases, one can use Delignc's deep result [De] to get better bounds. 
The general modulo idea to compute Z(X, T) consists of two steps. The first step 
is to compute Z{X, T) modulo various small primes (or prime powers). The second 
step is to use the Chinese remainder theorem to recover Z{X, T) from its various 
reductions. The second step is pretty easy and standard. The difficulty lies in the 
first step, namely, computing the reduction of Z(X, T) modulo various small primes 
or prime powers. 

Zeta functions of higher dimensional varieties over finite fields have been studied 
extensively from theoretic point of view, motivated by Weil's conjectures. Several 
theories are available but all of them are highly non-trivial and fairly difficult. They 
can be broadly classified as ^-adic methods and p-adic methods, where I is any prime 
number different from -p. In this section, we give a brief speculative discussion and 
perspective about the potential of these methods in computing zeta functions. No 
results and/or theorems are given in this section. 

^-adic methods. One can try to use Grothendieck's ^-adic trace formula [Gr] 
modulo £ m : 

2dim(X) 

Z(X,T)= Yl det(I - FT\H l c (X ® rq F g ,Z/l m Z)(- 1 Y~ 1 ( mod l m ), 

i=0 
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where H l c denotes the ^-adic etale cohomology with compact support and F is the 
geometric Frobenius map acting on H l c . Unfortunately, in the general case, one does 
not know how to explicitly construct the £-adic cohomology and its Frobenius map. 
In fact, for singular or open varieties, one does not even know any explicit bound for 
the dimension of the £-adic cohomology H l c . Thus, a major future research problem 
is to make the £-adic theory constructive and explicit. If successful, it would have 
dramatic consequence on the problem of counting the number of rational points of 
a variety over a finite field. 

In the special case of elliptic curves and abelian varieties, the £-adic cohomology 
can be constructed quite explicitly using torsion points and division polynomials. 
This immediately leads to the algorithms of Schoof for elliptic curves and Pila for 
abelian varieties. For a smooth curve X of genus g, one can use its Jacobian variety 
(an abelian variety) and Pila's algorithm to compute the zeta function Z(X,T) of 
the curve X, as done by Adleman-Huang. This algorithm is in general at least 
doubly exponential in g. In the case of hyperelliptic curve, the running time has 
been improved to be exponential in g. Thus it is still a very slow algorithm if g is 
large. If g is very small, this gives a good polynomial time algorithm. To conclude, 
even for smooth plane curves of large degrees over a small finite field, there is still 
no efficient algorithm to compute the zeta function. If X is singular, there is also 
a problem of resolving the singularities if one wants to use this approach. 

p-adic methods. There are various p-adic formulas for Z(X,T). All of them 
can be made to be explicit in some sense. The earliest one is due to Dwork, with 
generalizations by Reich-Monsky [Mo]. There are alsop-adic cohomological formu- 
las in terms of formal cohomology, crystalline cohomology and rigid cohomology 
[Ber] . We believe that some of these formulas and their variants can be very useful 
in computing the zeta function in the general case if the characteristic p is not too 
large. In particular, we expect it to be plausible to construct a p-adic algorithm 
to compute Z(X,T), which runs in polynomial time (in the input and output size) 
if X is sparse and the characteristic p is small (q can be large). This is exactly 
the case that is most useful in coding theory and cryptography. A weak corollary 
would be a polynomial time algorithm to compute N(X) for X of low degrees over 
a large finite field of small characteristic. Note that an elliptic curve is sparse and 
of low degree. If p is very large, we do not even know how to efficiently compute 
the reduction Z(X, T) modulo p. 

It is worthwhile to mention that the p-adic etale cohomological approach is not 
suitable for computing the whole zeta function. This is because the p-adic etale 
formula gives only the p-adic unit root part of the zeta function, not the whole 
thing, as conjectured by Katz [Kal] and proved in the zeta case by Etesse-Le Stum 
[ES]. 

Our general feeling is as follow. If p is small (q can be large), p-adic methods 
should be much more efficient. If p is large and the degree of X is small, then €-adic 
methods should be better assuming that the £-adic cohomology can be constructed 
explicitly. If both p and the degree are large, it is not clear if the p-adic methods or 
the f-adic methods would work very well. A combination of both p-adic methods 
and ^-adic methods could be useful in certain situations. 

For p-adic methods, the major advantage is that everything can be made to be 
explicit. This at least provides explicit algorithms which work in the general case 
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and which are faster in many cases than the trivial counting method. Another pos- 
sible advantage is that p-adic methods easily extend to L-functions of exponential 
sums with the same complexity bound. The drawback is that one does not have 
any flexibility to choose any other (small) prime to work with ( unlike the ^-adic 
case). Thus, if p is large, one is stuck and even the modulo p information is already 
something quite substantial. 

For £-adic methods, the major advantage is that one can always choose many 
small primes t ^ p to work with. One expects that it would be easier and more 
efficient to compute Z(X,T) modulo a small prime. However, the ^-adic cohomol- 
ogy is not explicit yet and thus one does not even know how to compute Z(X, T) 
using £-adic methods in general. Even if the £-adic methods are made explicit, the 
resulting algorithm could be very slow if the degree of the zeta function is large. 
For L-functions of exponential sums over a finite field of large characteristic p, it 
is not clear if the ^-adic methods would work well. This corresponds to the case of 
zeta functions of large degrees. 

In the rest of this paper, we discuss the weaker problem of computing the 
reduction of zeta functions modulo p or a power of p. In section 3, we consider 
the geometrically trivial case of zero dimensional hypersurfaces. In section 4, we 
consider the general problem of computing the zeta function modulo p for higher 
dimensional varieties focusing on hypersurfaces. Finally, in section 5, we compute 
Z(X, T) modulo a higher but small power of p. 

3. Zero dimensional hypersurfaces 

Let X be the zero dimensional hypersurface defined by a polynomial f(x) G 
¥ q [x] in one variable of degree d. Geometrically, X is trivial. It is a disjoint union of 
several closed points. Computing the zeta function in this simplest case is already 
non-trivial although it can be done efficiently as we shall see below. 

Let 

k 

f(x) = l[f i (x) a % a t >l 

i=l 

be the standard factorization of f(x) over ¥ q . Let di (1 < i < k) be the degree of 
the irreducible factor fi(x). An irreducible factor of f(x) of degree di corresponds 
to a closed point of X of degree di. The Euler product form of Z(X, T) shows that 

Question 3.1. Compute the zeta function Z(X,T) directly from f(x) without 
factoring f{x) over¥ q . 

By formula (3.1), an algorithm for factoring f{x) easily gives an algorithm for 
computing Z(X,T). It is interesting to note that the converse seems also true 
quite often. In this section, we shall discuss three elementary formulas for Z(X, T) 
modulo p, related to each other by duality and conjugacy. These lead to three 
simple polynomial time algorithms to compute Z(X, T). Each formula leads to an 
efficient algorithm for factoring f(x) if p is small. One formula leads to the classical 
algorithm of Bcrlckamp. Another formula leads to the more recent algorithm of 
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Niederreiter. The third one leads to a new algorithm, which has the advantage to 
extend to obtain higher dimensional zeta functions modulo p. 

3.1. Zeta function and Berlekamp's algorithm 

Let R = ¥ q [x]/(f) be the coordinate ring of the variety X/¥ q . This is an 
F g -vector space of dimension d. Let F be the g-th power Frobenius map on ¥ q [x] : 

F : h(x) — > h(x) q . 

It fixes the ideal (/) and thus induces a map on R. Using the Chinese remainder 
theorem and normal basis, one can easily prove the following result due to Petr 
(1937). 

Theorem 3.1.1. We have the congruence formula 

Z(X, T)- 1 = det(J - FT\R) ( mod p). 
The proof immediately gives the following result of Butler (1954): 

Theorem 3.1.2. Let R\(F) denote the subspace of R which is fixed by F. 
Namely, Ri{F) is the eigenspace of F at the eigenvalue 1. Then 

dim Wq Ri(F) = k, 

where k is the number of distinct irreducible factors of f(x) over ¥ q . 

Applying this result to f(x) over the extension field ¥ q j , we obtain the following 
result of Schwarz (1956). 

Theorem 3.1.3. Let k(j) be the number of distinct irreducible factors of f(x) 
over ¥ q j . Let Si be the number of distinct irreducible factors of degree i of f(x) 
over the ground field ¥ q . Then, we have 

dimF^O^') =k(j) 

d 

i=i 

where denotes the g.c.d. of the positive integers i and j . 

Let A be the d x d matrix whose ij entry is where 1 < i,j < d. As 

indicated by Schwarz, the matrix A is invertible. In fact, an explicit formula for 
the inverse matrix of A can also be found, see [GA]. 

The map F acting on R can be computed efficiently by repeated squaring. The 
above theorems show that all the k{i) and hence all the Sj (1 < i < d) can be 
computed in polynomial time. Thus, the zeta function 

i—l ^ } 

can be computed in polynomial time. It turns out that the subspace Ri(F) is also 
useful for the harder problem of factoring f(x) into irreducible factors over ¥ q . 
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Theorem 3.1.4. (Berlekamp [Be]). The eigenspace Ri(F) can be used to 
factor f(x) over ¥ q . 

The idea is as follows. By the Chinese remainder theorem, the eigenspace 
Ri (F) has a basis of the form 

r / \ f(x) , N f( X ) -, 

where yi(x) G R satisfies 

y^Tr^^ l{ mod *(*) 0< )- 

Jt\. x ) 1 

This basis shows that if h\(x) and fi2(x) arc two elements of Ri{F) which are 
linearly independent over ¥ q , then 

/(*) = ]J (/(*). hi(x) - ch 2 {x)) (3.1.1) 

gives a non-trivial factorization of /(x). This algorithm clearly runs in polynomial 
time if q is small. It can be extended to the case when p is small (q can be large). 
At present, no deterministic polynomial time algorithm is known to factor f(x) 
in general if p is large even under the assumption of the Generalized Riemann 
Hypothesis. 

A subspacc R\ of R is called admissible if any two linearly independent ele- 
ments hi and h 2 of R\ satisfy (3.1.1). It is clear that if an admissible subspace of R 
can be constructed efficiently, then one can proceeds in the same way to construct 
a new factorization algorithm. In the next two subsections, we describe two more 
such admissible subspaces and hence two more such algorithms for factoring poly- 
nomials. Admissible subspaces are studied in some details in Lee-Vanstone [LV]. In 
particular, the congruence formula in section 3.3 gives a new admissible subspace 
and hence answers a question in [LV]. 

3.2. Zeta function and Niederreiter's algorithm 

Let tjj q be the F 9 -linear operator on ¥ q [x] defined by 

*<*■>-{ f' T- 

l 0, otherwise. 

This is a one sided inverse of the Frobenius map. It arises in various contexts 
such as Dwork's operator in p-adic theory of zeta functions, Cartier's operator in 
algebraic geometry and Hecke's U-operator in modular forms. 

Let i^ 9-1 ) denote the (q — l)-th Hasse derivative on ¥ q [x] defined by 



x u ~( q ~ l ) . 



u 

Let D be the F g -linear differential operator on ¥ q [x] : 

D = iP q oH (q - 1) o/fo- 1 ), 
where is the multiplication operator. Since 

<P°f q = f° f q ° = o f\ 
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the ideal (/) of ¥ q [x] is stable under the action of D. Thus, the operator D induces 
a well defined map on R = ¥ q [x]/(f). The subspace R\{D) of R fixed by D has 
the admissible basis 

where f-(x) denotes the derivative of f(x). It follows that we have 

Theorem 3.2.1. (Niederreiter [Ni]). The eigenspace R\(D) can be used to 
factor f(x) over W q in a way similar to Berlekamp's algorithm. 

Using the Chinese remainder theorem, normal basis and basic properties of the 
operators ip q an d H^" 1 ^, it is elementary to prove directly 

Theorem 3.2.2. We have the congruence formula 

Z(X, T)- 1 = dct(7 - DT\R) ( mod p). 

As before, the eigenspaces R\{D : >) can be used to determine the integers Sj and 
hence gives a polynomial time algorithm to compute the zeta function Z(X,T). 

3.3. A new congruence formula 

Let G be the composition of i\) q and the multiplication operator f^ 1 acting on 
¥ q [x}: 

G = VW (9_1) - 

Again, the ideal (/) of ¥ q [x] is stable under the action of G. Thus, the operator 
G induces a well defined map on R = ¥ q [x}/(f). Using the Chinese remainder 
theorem, normal basis and basic property of the operators i[) g , we deduce 

Theorem 3.3.1. Assume that /(0) ^ 0. Then, 

Z(X, T)- 1 = det(J - GT\R) ( mod p). 

Note that this result is false in general if /(0) = 0. Similarly, the eigenspaces 
R\(G : ') can be used to determine the integers and hence yields a polynomial 
time algorithm to compute the zeta function Z(X, T). For /(0) ^ 0, the eigenspace 
R\{G) has the admissible basis 

{xm m y ..., xmx) m } . 

Thus, we obtain 

Theorem 3.3.2. The eigenspace Ri(G) can be used to factor f(x) over¥ q in 
a way similar to the algorithms of Berlekamp and Niederreiter. 



3.4. Duality and conjugacy 

The above three operators are closely related to each other. Using the defini- 
tions of D and G, one can easily check that they are conjugate to each other via 
the element x if x is invertible. Namely, we have 

Theorem 3.4.1. If /(0) ^ 0, then D = x^Gx. 

The relation to the Frobenius map is given by duality. The proof is elementary. 
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Theorem 3.4.2. There are two non-singular pairings on R x R such that F 
and D (resp. F and G) are dual with respect to the first (resp. the second) pairing. 

The advantage of the operator E = ip q o fi^ 1 is that it extends to higher 
dimensional varieties. We shall discuss this in next section. 

4. Higher dimensional varieties 

For simplicity, we restrict to the case of a hypersurface. The method works for 
an arbitrary variety. Thus, let X be the affine hypersurface defined by a polynomial 
f(xi, • • • , x n ) over ¥ q of total degree d in n variables. Let R(d) be the F 9 -vector 
space 

R(d) = (xf •x„F 9 [xi, • • • ,x n ])< d . 
Namely, R(d) is the F g -vector space generated by those monomials of degree at 
most d and divisible by the product x\ ■ ■ ■ x n . One computes that 

dim F q R(d) = 

The operator \p q extends easily to several variable case. We shall see that the 
operator ip q o f' 1 can be used to compute the zeta function Z(X, T) modulo p. 

PROPOSITION 4.1. The operator ip q o j s stable on the subspace R(d). 

Proof. Let h e R(d). Then, h is a polynomial of degree at most d. One 
checks that the degree of f q ~ 1 h is at most d(q — 1) + d = dq. Thus, the degree of 
ipqif 9-1 h) is at most d. Furthermore, if h is divisible by x\ ■ ■ ■ x n , then ip q (f q ~ 1 h) 
is also divisible by x\ ■ ■ ■ x n . This proves that ■0g(/ <? ~ 1 ^) G R(d). 

The following simple congruence formula for zeta functions can be easily proved 
using the reduction of the Dwork trace formula. It is implicit in section 7 of [Wa2] . 

Theorem 4.2. We have the congruence formula 

Z(X,T) { - 1)n = det(J - {ijj q o p-^TlRid)) ( mod p). 

In the projective similar congruence formula but with a much harder 

proof is given by Katz [Ka2]. Very general congruence formulas but acting on 
infinite dimensional space can be found in [TW1] and [TW2]. Using Theorem 4.2, 
we get 

Corollary 4.3. The zeta function Z(X, T) modulo p can be computed in time 
that is a polynomial in ( ) logq (the input and output size) if p is small and n is 
fixed. 

This follows directly from Theorem 4.2 if q is small. If q is large but p is small, 
one needs to use a little semi-linear algebra and Galois theory. If p is large, we do 
not get a polynomial time algorithm. In computing the high power f 1 ^ 1 , we have 
to expand it without reduction and the size simply gets larger and larger if p is 
large. 

5. Higher power congruences 

The congruence result in section 4 can be improved to get finer information by 
using congruences modulo a higher power of p. For this purpose, we need to use 
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p-adic liftings. Let K be an unramificd finite extension of Q p with residue field 
F g . Let Ok be the ring of integers in K. For a positive integer m, Let O m be the 
residue class ring OkHp™ 1 ), which is a finite ring of q m elements. 

We still restrict to the case of a hypersurface. For simplicity of description, we 
shall only consider those solutions with x\ ■ ■ ■ x n ^ 0. Thus, let X be the affinc 
hypersurface in the torus G^/¥ q defined by a polynomial f(xi, • • • , x n ) over ¥ q of 
total degree d in n variables. We assume that the polynomial / has already been 
lifted to O m for some m > 0. We shall consider the question of computing Z(X, T) 
modulo p m . 

Let R m ,d be the free O r „-module: 

Rm,d = Cm [xi j ' ' ' ; ^n] <dp m_1 • 

Namely, R m ,d is the free O m -module generated by those monomials of degree at 
most dp™ -1 . One computes that 

(dp m - l +n 
dim 0m R m ,d = I n 

The operator tp q is defined as before. It is easy to check that we have 

Proposition 5.1. The operator %f) q o fii- 1 )?™ 1 is stable on the finite dimen- 
sional free O m -module R m ,d- 

The operator tp q o f(i- 1 )p m 1 modulo p m is clearly independent of the choice of 
the lifting of / to O m . The following simple congruence formula for zeta functions 
can be proved using the reduction modulo p m of the Dwork trace formula. 

Theorem 5.2. We have the congruence formula 
{ Z(Gn T T) ){ ~ ir = II det ( J - g*(^ ° f^^mRmJ- 1 ^ ( mod p m ). 

■ m ' ^ i=0 

Since the zeta function of the n-torus is very simple, we are reduced to 
computing det(7 — (ip q o /(9 _1 )p m )T\R m ,d) modulo p m . Using the above formula, 
we can show 

COROLLARY 5.3. Z(X, T) modulo p m can be computed in polynomial time if 
p m is small and n is fixed. 

This follows from Theorem 5.2 if q is also small. If q is large but p m is small, 
it involves a little extra work. If p rn is large, we do not get a polynomial time 
algorithm as the dimension of R m ,d is already exponential. 

The approach we take in this section is the most elementary one. It has the 
advantage to be a polynomial time algorithm for any hypersurface as long as p m is 
small and as long as we only want modulo p m information for Z(X,T). We shall 
consider another deeper p-adic approach in a future article, which seem to work 
well in some interesting cases even when we want the full information for Z(X, T). 
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